Dynamic content reordering

ABSTRACT

In various example embodiments, a system and method for dynamically ordering content and discrete content segments are presented. A set of discrete content segments is received having a first order for distributing the set of discrete content segments during a display of a media stream. Event data is received. The event data is representative of a set of events depicted by the media stream. A set of event instances is determined from the event data representing the set of events depicted by the media stream. A second order is generated for the set of content segments based on the set of event instances.

PRIORITY

This application is a continuation of and claims the benefit of priorityto U.S. patent application Ser. No. 14/738,670, filed on Jun. 12, 2015,which is hereby incorporated by reference herein in its entirety.

TECHNICAL FIELD

Embodiments of the present disclosure relate generally to dataprocessing and, more particularly, but not by way of limitation, todynamic content reordering.

BACKGROUND

Conventionally, advertisements may be placed within a media stream basedon a bidding process, whereby advertisers bid on predeterminedadvertisement slots. Pricing for advertisement slots may vary throughouta given media stream and be influenced by a time of day at which theslot occurs, a program being transmitted within the media stream, and arelative time of year for a broadcast season. Sections of a media streammay include national advertisement portions and local advertisementportions. National advertisement portions of a media stream may bepartitioned into slots for advertisements to be displayed across a givennation in which the media stream is broadcast or otherwise distributed.Local advertisement portions may be partitioned into slots foradvertisements limited to a specific locality in which the media streamis broadcast. As such, once a media stream reaches a specific locality,the local advertisement slots may be sold using a similar process asoutlined generally above.

BRIEF DESCRIPTION OF THE DRAWINGS

Various ones of the appended drawings merely illustrate exampleembodiments of the present disclosure and cannot be considered aslimiting its scope.

FIG. 1 is a block diagram illustrating a networked system, according tosome example embodiments.

FIG. 2 is a block diagram illustrating components of a content orderingsystem suitable for dynamically generating an order for discrete contentsegments within a content stream.

FIG. 3 is a flow diagram illustrating operations of a method fordynamically generating an order for discrete content segments usinghardware modules of the content ordering system of FIG. 2, according tosome example embodiments.

FIG. 4 is a flow diagram illustrating operations of a method fordynamically generating an order for discrete content segments usinghardware modules of the content ordering system of FIG. 2, according tosome example embodiments.

FIG. 5 is a flow diagram illustrating operations of a method fordynamically generating an order for discrete content segments usinghardware modules of the content ordering system of FIG. 2, according tosome example embodiments.

FIG. 6 is a block diagram illustrating an example of a softwarearchitecture that may be installed on a machine, according to someexample embodiments.

FIG. 7 illustrates a diagrammatic representation of a machine in theform of a computer system within which a set of instructions may beexecuted for causing the machine to perform any one or more of themethodologies discussed herein, according to an example embodiment.

The headings provided herein are merely for convenience and do notnecessarily affect the scope or meaning of the terms used.

DETAILED DESCRIPTION

The description that follows includes systems, methods, techniques,instruction sequences, and computing machine program products thatembody illustrative embodiments of the disclosure. In the followingdescription, for the purposes of explanation, numerous specific detailsare set forth in order to provide an understanding of variousembodiments of the inventive subject matter. It will be evident,however, to those skilled in the art, that embodiments of the inventivesubject matter may be practiced without these specific details. Ingeneral, well-known instruction instances, protocols, structures, andtechniques are not necessarily shown in detail.

Conventional advertisement schemes lack an ability to direct content todisplay locations within a locality and specific viewing audienceswithin the locality or the display location based on a composition andidentifying characteristics of subjects viewing a media stream in whichthe content appears. Further, conventional content delivery schemes lackan ability to contextually order content within the media stream so asto display content at a favorable time during the transmission of themedia stream or to an audience at a time in which the audience is mostlikely to be susceptible to or positively receive the content.

In various example embodiments, systems and methods for a contentordering system for dynamically ordering or reordering discrete contentsegments within a content stream or media stream are presented. Forexample, a set of discrete content segments can be a set ofadvertisements to be partitioned into a set of content segment slotswithin a broadcast media stream (e.g., a scripted television program, alive television broadcast, a sporting event, etc.). The content orderingsystem disclosed can receive the set of content segments along with aset of criteria, factors, or contextual information for the contentsegments indicating a desired time, event, or audience mood at which thecontent segments are to be displayed. In addition, the content orderingsystem can receive sensor data from devices broadcasting the mediastream or sensors associated with the device broadcasting the mediastream in order to determine characteristics, attitudes, attentionlevels, emotional reactions, and the like for subjects perceived by thesensors and consuming the media stream.

For example, a content producer may produce a content segment linkingthe content producer with a specific sports franchise or team. Thecontent producer may place a bid to display the content segment withinthe media stream depicting a game being played by the team. The bid mayinclude a frequency for displaying the content segment and a context fordisplaying the content segment (e.g., display the content segment afterthe high point of the first quarter for the team, display the contentsegment three times during the game but not after a play or call that isnegative for the team, display the content segment after a touchdown bythe team, etc.). The content ordering system may determine an order forthe content segment and other content segments within a content stream.For example, the order of the content segment may be based on eventswithin the media stream (e.g., the score of the game for either team,etc.), the subjects consuming the media stream (e.g., an order based ondetecting that the subjects are fans of the team relating to the contentsegment), or both. The content ordering system may also reorder thecontent segments based on changes in the program, an occurrence of oneor more specified events (e.g., events with a probability of occurrencebut not a guarantee of occurrence), changes in reactions or attention ofthe subjects consuming the media stream, or combinations thereof. Thecontent producer may have the content segment displayed at times whenthe subjects are more likely to positively receive the content segments,more likely to view and be attentive to the display of the contentsegment, and other factors positively affecting the reception of thecontent segment by the audience.

With reference to FIG. 1, an example embodiment of a high-levelclient-server-based network architecture 100 is shown. A networkedsystem 102, in the example forms of a network-based marketplace orpayment system, provides server-side functionality via a network 104(e.g., the Internet or a wide area network (WAN)) to one or more clientdevices 110. FIG. 1 illustrates, for example, a web client 112 (e.g., abrowser, such as the Internet Explorer® browser developed by Microsoft®Corporation of Redmond, Wash. State), a client application 114, and aprogrammatic client 116 executing on the client device 110.

The one or more client devices 110 may comprise, but are not limited to,mobile phones, desktop computers, laptops, portable digital assistants(PDAs), smart phones, tablets, ultra books, netbooks, laptops,multi-processor systems, microprocessor-based or programmable consumerelectronics, game consoles, set-top boxes, televisions, smarttelevisions, or any other communication device that a user may utilizeto access the networked system 102. In some embodiments, the clientdevice 110 may comprise a display module (not shown) to displayinformation (e.g., in the form of user interfaces). The client device110 may also include or be associated with a set of sensors 118. Infurther embodiments, the set of sensors 118 of the client device 110 maycomprise one or more of touch screens, accelerometers, gyroscopes,cameras, microphones, global positioning system (GPS) devices, and soforth. The client device 110 may be a device of a user that is used toreceive a media stream (e.g., a broadcast of a television program, atransmission of an online program, a transmission of a streamingentertainment service, etc.) within the networked system 102 ortransmitted to the client device 110 by a broadcast system 136 or athird party server 130. In one embodiment, the networked system 102 is anetwork-based marketplace that responds to requests for productlistings, publishes publications comprising item listings of productsavailable on the network-based marketplace (e.g., media products, mediastreams, and downloadable media content), and manages payments formarketplace transactions. One or more users 106 may be a person, amachine, or other means of interacting with the client device 110. Inembodiments, the user 106 is not part of the network architecture 100,but may interact with the network architecture 100 via the client device110 or another means. For example, one or more portions of the network104 may be an ad hoc network, an intranet, an extranet, a virtualprivate network (VPN), a local area network (LAN), a wireless LAN(WLAN), a wide area network (WAN), a wireless WAN (WWAN), a metropolitanarea network (MAN), a portion of the Internet, a portion of the PublicSwitched Telephone Network (PSTN), a cellular telephone network, awireless network, a WiFi network, a WiMax network, another type ofnetwork, or a combination of two or more such networks.

The client device 110 may include one or more applications (alsoreferred to as “apps”) such as, but not limited to, a web browser,messaging application, electronic mail (email) application, e-commercesite application (also referred to as a marketplace application), andthe like. The e-commerce site application may enable transmission of amedia stream (e.g., an online movie, a video game, a video, a televisionprogram, etc.) and purchase of items (e.g, physical items, digitalitems, media items, etc.). In some embodiments, if the e-commerce siteapplication is included in the client device 110, then this applicationis configured to locally provide a user interface and at least somefunctionalities of an e-commerce site and to communicate with thenetworked system 102, on an as needed basis, for data and/or processingcapabilities not locally available (e.g., access to a database of itemsavailable for sale, to authenticate a user, to verify a method ofpayment, access to a database of advertising content, access to adatabase or service for streaming media content, etc.). Conversely ifthe e-commerce site application is not included in the client device110, the client device 110 may use its web browser to access thee-commerce site (or a variant thereof) hosted on the networked system102.

The one or more users 106 may be a person, a machine, or other means ofinteracting with the client device 110. In example embodiments, the user106 is not part of the network architecture 100, but may interact withthe network architecture 100 via the client device 110 or other means.For instance, the user 106 provides input (e.g., touch screen input oralphanumeric input) to the client device 110 and the input iscommunicated to the networked system 102 via the network 104. In thisinstance, the networked system 102, in response to receiving the inputfrom the user 106, communicates information to the client device 110 viathe network 104 to be presented to the user 106. In this way, the user106 can interact with the networked system 102 using the client device110.

An application program interface (API) server 120 and a web server 122are coupled to, and provide programmatic and web interfaces respectivelyto, one or more application servers 140. The application servers 140 mayhost one or more publication systems 142, payment systems 144, andcontent ordering systems 150, each of which may comprise one or moremodules or applications, and each of which may be embodied as hardware,software, firmware, or any combination thereof. The application servers140 are, in turn, shown to be coupled to one or more database servers124 that facilitate access to one or more information storagerepositories or database(s) 126. In an example embodiment, the databases126 are storage devices that store information to be posted (e.g.,publications or listings) to the publication system 142. The databases126 may also store digital item information, in accordance with exampleembodiments.

Additionally, a third party application 132, executing on third partyserver(s) 130, is shown as having programmatic access to the networkedsystem 102 via the programmatic interface provided by the API server120. For example, the third party application 132, utilizing informationretrieved from the networked system 102, supports one or more featuresor functions on a website hosted by a third party. The third partywebsite, for example, provides one or more promotional, marketplace, orpayment functions, streaming media content, or downloadable mediacontent that are supported by the relevant applications of the networkedsystem 102.

The publication systems 142 may provide a number of publicationfunctions and services to users 106 that access the networked system102. The payment systems 144 may likewise provide a number of functionsto perform or facilitate payments, transactions, transmission of mediastreams, and downloading of media content. While the publication system142 and payment system 144 are shown in FIG. 1 to both form part of thenetworked system 102, it will be appreciated that, in alternativeembodiments, each system 142 and 144 may form part of a payment servicethat is separate and distinct from the networked system 102. In someembodiments, the payment systems 144 may form part of the publicationsystem 142.

The content ordering system 150 may provide functionality operable todynamically order or reorder discrete content segments within one ormore streams of content based on sensor data and events within a mediastream. For example, the content ordering system 150 may receive acontent stream including a set of discrete content segments from thedatabases 126, the third party servers 130, the publication system 142,the broadcast system 136, or other sources. The content ordering system150 may additionally receive sensor data from a display location for thecontent stream. In some instances, the content ordering system 150receives event data of a media stream from the broadcast system 136.

In some example embodiments, the content ordering system 150 may analyzethe sensor data to order the discrete content segments of the contentstream based on the sensor data. In some instances, where the set ofdiscrete content segments is provided in a first order for transmissionwithin the content stream, the content ordering system 150 can cause areordering to present the set of discrete content segments in a secondorder based on one or more of the sensor data and the events within themedia stream. As more sensor data is received and more events occurwithin the media stream, the content ordering system 150 can furtherrefine and subsequently redefine the order in which the set of discretecontent segments is presented within the content stream. In some exampleembodiments, the content ordering system 150 may communicate with thepublication systems 142 (e.g., accessing discrete content segments), thepayment system 144, and the broadcast system 136. In an alternativeembodiment, the content ordering system 150 may be a part of thepublication system 142 or the broadcast system 136.

Further, while the client-server-based network architecture 100 shown inFIG. 1 is a client-server architecture, the present inventive subjectmatter is of course not limited to such an architecture, and couldequally well find application in a distributed, or peer-to-peer,architecture system, for example. The various publication system 142,payment system 144, and content ordering system 150 could also beimplemented as standalone software programs, which do not necessarilyhave networking capabilities.

The web client 112 may access the various publication and paymentsystems 142 and 144 via the web interface supported by the web server122. Similarly, the programmatic client 116 accesses the variousservices and functions provided by the publication and payment systems142 and 144 via the programmatic interface provided by the API server120. The programmatic client 116 may, for example, be a sellerapplication (e.g., the Turbo Lister application developed by eBay® Inc.,of San Jose, Calif.) to enable sellers to author and manage listings onthe networked system 102 in an off-line manner, and to performbatch-mode communications between the programmatic client 116 and thenetworked system 102.

FIG. 2 is a block diagram illustrating components of the contentordering system 150, according to some example embodiments. The contentordering system 150 is shown as including an access module 210, areceiving module 220, a detection module 230, an engagement module 240,an order module 250, a presentation module 260, and a communicationmodule 270, all configured to communicate with each other (e.g., via abus, shared memory, or a switch). Any one or more of the modulesdescribed herein may be implemented using hardware (e.g., at least oneprocessor of a machine) or a combination of hardware and software. Forexample, any module described herein may configure a processor (e.g.,among one or more processors of a machine) to perform one or more of theoperations for which that module is designed. Moreover, any two or moreof these modules may be combined into a single module, and the functionsdescribed herein for a single module may be subdivided among multiplemodules. Furthermore, according to various example embodiments, modulesdescribed herein as being implemented within a single machine, database126, or device may be distributed across multiple machines, databases126, or devices.

The access module 210 accesses, receives, or otherwise obtains data fromone or more systems, databases, or components. The access module 210 isconfigured to access a set of discrete content segments. In someembodiments, the set of discrete content segments can be stored on thedatabase 126 of the network-based publication system 142. In otherembodiments, the access module 210 may access the set of discretecontent segments stored on a database 133 associated with the broadcastsystem 136. In some instances, the access module 210 accesses thediscrete content segments at a time prior to commencement of the mediastream from the broadcast system 136 or at the time of transmission ofthe media stream by the broadcast system 136.

The receiving module 220 receives event data of the media stream fromthe broadcast system 136. In some embodiments, the receiving module 220receives sensor data from a set of sensors at a display location for themedia stream. The receiving module 220 can receive sensor data from aplurality of sets of sensors located at a plurality of different displaylocations. Each set of sensors of the plurality of sensors may transmita discrete stream or set of sensor data to be received by the receivingmodule 220. In some embodiments, the receiving module 220 may receivethe sensor data in a continuous or semi-continuous stream during a timeperiod in which the media stream is being transmitted by the broadcastsystem 136.

In some instances, the receiving module 220 may receive bursts orperiodic sensor data. The bursts or periodic sensor data may beindicative of a sensor acquiring the sensor data during set periods oftime (e.g., the sensor activates every one, three, or five minutes),upon receiving an input (e.g., the sensor is triggered by motion orsound), as discrete packets (e.g., the sensor takes a single image orseries of images), or by any other suitable method. The sensor datareceived from each display location of the plurality of differentdisplay locations may be distinct from sensor data received from otherdisplay locations, and may cause differing orders to be imposed on thecontent stream, as discussed in more detail below.

The detection module 230 may be understood as a module configured toparse the media stream. The detection module detects a set of eventinstances from the event data depicted by the media stream. In someembodiments, the detection module 230 may detect the set of eventinstances from image recognition or other processing of the mediastream. The detection module 230 may detect the set of event instancesfrom metadata (e.g., closed captioning data, announcer commentary, orother sources of metadata) associated with or included within the mediastream.

The engagement module 240 generates a set of emotion indicators based onthe sensor data. The set of emotion indicators represents an emotionalstate of one or more subjects perceived by the set of sensors at adisplay location. In some embodiments, the set of emotion indicators maybe based on sensor data representing one or more indicators selectedfrom a group consisting of a facial feature, a body position (e.g.,posture, proximity of a body to the sensor, and the like), a volume ofconversation, and a content of conversation.

In some embodiments, the engagement module 240 generates a set ofattention factors based on the sensor data. The set of attention factorsrepresents an estimation of a level of attention paid, by one or moresubjects perceived by the set of sensors, to the media stream. In someembodiments, the set of attention factors may be based on sensor datarepresenting one or more indicators selected from a group comprisingsubject proximity to a sensor, a number of subjects perceived by thesensor (e.g., the number of subjects the sensor currently perceives), asubject facial orientation, a subject body orientation, a subject facialexpression, a volume level of the subjects, a conversation indicator ofthe subjects (e.g., content of the conversation, direction of theconversation, etc.), and other characteristics or physical positioningof the one or more subjects perceived by the sensor.

The order module 250 is configured to generate an order for the set ofdiscrete content segments of the content stream. In some embodiments,the order module 250 generates an order where the set of discretecontent segments includes no prior order. In some instances, the set ofdiscrete content segments includes a first order when accessed by theaccess module 210. In embodiments where the set of discrete contentsegments includes a first order, the order module 250 may generate asecond order for the set of discrete content segments. The order module250 may determine the order based on the set of event instances. In someembodiments, the order module 250 may determine the order based on theset of event instances and a set of ordering factors for the set ofdiscrete content segments.

Where the receiving module 220 receives sensor data for a displaylocation, the order module 250 can generate the order based on one ormore of the set of event instances, the set of ordering factors, and thesensor data. In these instances, the order module 250 may generate theorder for the display location associated with the sensor data. In thisway, the order module 250 may generate a distinct order for each displaylocation for which the receiving module 220 receives sensor data.Further, in some embodiments, the order module 250 may generate theorder based on one or more of the set of emotion indicators, the set ofattention factors, the set of event instances, the set of orderingfactors, a set of engagement indicators, and the sensor data.

The presentation module 260 causes presentation of the set of discretecontent segments within the content stream according to the orderdetermined by the order module 250. For example, the presentation module260 can generate the content stream, and the included set of discretecontent segments, on a display device associated with the client device110. The presentation module 260 may cause presentation of the contentstream by transmitting the content stream and appropriate displayinstructions to the client device 110, displaying the content stream onthe client device 110, generating one or more screens or web pages, orany other suitable method. Although embodiments are discussed withreference to a display device, it should be understood that thepresentation module 260 can cause presentation of the content stream andthe set of discrete content segments in any suitable manner orcombination of manners (e.g., audio only, video only, still pictures,audio and video, still pictures and audio, etc.).

The communication module 270 enables communication among the clientdevice 110, the content ordering system 150, the publication system 142,and one or more external systems (e.g., the payment system 144, thebroadcast system 136, or other systems). In some example embodiments,the communication module 270 can enable communication among the accessmodule 210, the receiving module 220, the detection module 230, theengagement module 240, the order module 250, and the presentation module260. The communication module 270 can be a hardware implemented module,a software implemented module, or a combination thereof, as described inmore detail below. For example, the communication module 270 can includecommunications mechanisms such as an antenna, a transmitter, one or morebuses, and other suitable communication mechanisms capable of enablingcommunication between the modules 210-260, the client device 110, thecontent ordering system 150, the publication system 142, the paymentsystem 144, or the broadcast system 136.

FIG. 3 is a flow chart of operations of the content ordering system 150in performing a method 300 of ordering content for transmission within acontent stream, according to some example embodiments. Operations in themethod 300 may be performed by the content ordering system 150, usingmodules described above with respect to FIG. 2. In some embodiments, theoperations of the method 300 can be performed using the modules of FIG.2, where portions of the modules may be implemented on client devices,such as the client device 110, or on the broadcast system 136.

In operation 310, the access module 210 accesses a set of discretecontent segments for display in a media stream. The access module 210may access the set of discrete content segments stored on the database126 via the communication module 270. In some embodiments, the set ofdiscrete content segments may be stored in a database of the broadcastsystem 136, and accessed using one or more requests and responses of theaccess module 210 transmitted by the communication module 270. Theaccess module 210 may also access the discrete content segments acrossmultiple databases on one or more systems. In these embodiments, theaccess module 210 may transmit the discrete content segments, accessedon different systems, to a central database (e.g., the database 126) tocreate the set of discrete content segments, or to another module forcompletion of the method 300.

Each discrete content segment of the set of discrete content segmentsmay include a content and a set of ordering factors. The set of orderingfactors may be understood to be a set of conditions guiding ordering anddisplay of the discrete content segment. The ordering factors may bedetermined by the producer or owner of the discrete content segment, athird party entity (e.g., FCC), or any other entity having rights todistribute or limit distribution of the discrete content segment. Forexample, the discrete content segment may be advertisement.

In some embodiments, the set of ordering factors may include a set ofevent factors understood to be events (e.g., a baseball player hitting ahome run, a quarterback in a football game being sacked, a musical guestplaying on a live broadcast of a comedy or variety show, a dramaticmoment of a scripted television program, etc.), actions, keywords (e.g.,keywords spoken by a person during a show, a keyword displayed on aclosed caption system, a keyword displayed in a scrolling or staticchyron displayed on the client device 110, etc.), time periods (e.g.,entering a fourth quarter of a sporting event), or other suitable eventfactors. In some embodiments, the actions can be directly related to themedia stream or indirectly related to the media stream.

Actions directly related to the media stream may be understood to beactions being displayed, spoken, or otherwise happening within the mediastream. For example, the directly related actions could include a playersignaling a time-out, a referee stopping a game to review a play, anannouncer commenting on a play, and the like. By way of further example,actions directly related to the media stream can include aspects of themedia stream perceived by image, video, or speech recognition systems.Exemplary aspects of the media stream can include referee signals;banners (e.g., chyrons) indicating scoring, actions of players, or thelike; referee signals (e.g., as perceived by image/video/speechrecognition systems); and other aspects directly related to the mediastream or depicting portions of the media stream. In some instances,image or video recognition systems may include image recognition modulescapable of receiving images, video, or live video using image sensorsand processing the images, video, or live video to identify patterns,words, motions, or other items to recognize the aspects of the mediastream. The audio recognition system can include audio recognitionmodules (e.g., speech recognition modules, noise recognition modules,etc.) along with one or more microphone or other audio sensor toidentify and discern between types or wave forms of audio data torecognize the aspects of the media stream. In some embodiments,indirectly related actions can include actions of one or more subjectsperceived by the sensors, as will be explained in more detail below.

Indirectly related actions could include, for example, one or moresubjects viewing the media stream cheering for an event, shouting towardthe client device 110, jumping, sitting down, turning away from theclient device 110, or any other suitable action. The event factors mayindicate one or more events of the set of events depicted in the mediastream after which a discrete content segment is to be displayed.

In some instances, the set of ordering factors may include a set ofdistribution factors understood to be conditions for inclusion of adiscrete content segment into the content stream. These distributionfactors may be independent of the set of events within the media stream.For example, the distribution factors may be indicative of arelationship between the producer of a discrete content segment and abroadcast or publishing entity on whose signal the discrete contentsegment is to be transmitted. In some embodiments, the set ofdistribution factors may be selected from a group comprising anassociation among one or more discrete content segments of the set ofdiscrete content segments, a distribution total for the discrete contentsegment, a bid price of the discrete content segment, or a total pricepaid for a subset of related discrete content segments.

In operation 320, the receiving module 220 receives event datarepresentative of the media stream. The event data may be understoodbroadly as a content of the media stream, or more specifically, aspectsof the content of the media stream which may be contextually related toor capable of affecting reception of a discrete content segment (e.g.,an advertisement) by an audience of the media stream. In someembodiments, the event data may be an aggregation of common and uncommonoccurrences within a portion of the media stream (e.g., a sportingevent, a live television event, a scripted program) which may have aneffect on the reception of the discrete content segment (e.g., anadvertisement) and may be difficult to predict prior to their occurrencewithin the media stream. The receiving module 220 may receive the eventdata by receiving the media stream from the broadcast system 136, forexample. In some embodiments, all or a portion of the event data may betransmitted along with the media stream as metadata associated with themedia stream. The metadata may be a reflection of the event data, suchas closed captioning data, announcer commentary, or other sources ofmetadata.

The receiving module 220 may receive the event data directly from thebroadcast system 136, receiving the transmission of the media stream viaa network, a broadcast transmission, or any other suitable communicationmethod. In some embodiments, at least a portion of the receiving module220 may be implemented in the broadcast system 136, such that thereceiving module 220 directly receives the media stream and passes theevent data from the media stream back to the content ordering system150. In some instances, the receiving module 220 may receive the eventdata through the client device 110, such that once the client device 110receives the media stream, the client device 110 transmits all or aportion of the media stream to be received by the receiving module 220.Although a few methods of receiving the event data have been disclosed,it will be understood that other suitable methods of receiving one ormore of the media stream and the event data may also be used.

In operation 330, the detection module 230 detect a set of eventinstances of from the event data depicted by the media stream. An eventinstance can be understood as a discrete instance of a set of possibleoccurrences within the media stream. For example, in a baseball game,the event data may include a plurality of runs or pitches, while anevent instance can be a run amounting to a grand slam, a pitch whichstrikes a batter out, a pitch by a specified team, or the like. In someembodiments, the event instance may be one or more occurrences which actas a precursor or possible precursor to a desired event or occurrence.For example, the event instance in a sporting event, such as a footballgame, can be a series of successive downs by a team which place the teamin a scoring position.

In operation 340, the order module 250 generates an order for the set ofdiscrete content segments for transmission within the content stream.The order may be generated based on the set of event instances and theset of ordering factors for each discrete content segment of the set ofdiscrete content segments. The order module 250 may generate the orderbased on a weighted function receiving the set of ordering factors(e.g., the set of event factors and the set of distribution factors) foreach of the discrete content segments and the set of event instances asinputs to the weighted function. In some embodiments, the set ofdiscrete content segments may be provided in a predetermined first orderfor presentation within the content stream. In these instances, theorder module 250 may generate a second order for the set of discretecontent segments.

In some embodiments, the order module 250 determines the order of theset of discrete content segments using one or more segment matrices. Theone or more segment matrices may plot each discrete content segmentwithin a segment matrix, for example, by plotting one or more of theordering factors against a monetary value. The monetary value mayrepresent a fixed price paid by the producer of the discrete contentsegment, a bid for placement of the discrete content segment, or anyother monetary value the producer may promise to pay for placement ofthe discrete content segment within the content stream or the mediastream. As discussed, the one or more segment matrices may provide acost related context to the set of discrete content segments used asinput for the order module 250.

For example, upon receiving an event instance of a touchdown in afootball game, the order module 250 may determine one or more discretecontent segments from the set of discrete content segments having anordering factor of the set of ordering factors relating to a touchdown.The order module 250 may order the set of discrete content segments suchthat the discrete content segments having an ordering factor relating toa touchdown (e.g., the discrete content segment is to be displayed aftera touchdown) are placed in a relative order. For example, the discretecontent segment may be placed toward the front of the order or towardthe end of the order when the audience is more likely to be viewing theclient device 110. The order module 250 may determine the position ofthe set of discrete content segments, from their relative order, usingthe monetary value associated with the discrete content segments. Forexample, the order module 250 may place the discrete content segmentshaving a higher monetary value into a more prominent position within therelative order. In some embodiments, the order module 250 may prioritizethe ordering factor over the monetary value in order to generate abetter contextual fit between the event instance and the set of discretecontent segments.

Further, in some instances, the order module 250 may prioritize adiscrete content segment within the order based on a second segmentmatrix. For example, the order module 250 may determine a first orderbased on the event instance, the ordering factors, and the monetaryvalue. The order module 250 may then determine, using the second segmentmatrix, that two discrete content segments are to be played back-to-backin an order or within a predetermined number of discrete contentsegments from one another. The order module 250 may then reorder thefirst order into a second order to satisfy the ordering factor of thesecond segment matrix.

In some embodiments, the method 300 may include operation 350. Inoperation 350, the presentation module 260 causes presentation of theset of discrete content segments within the media stream according tothe order determined by the order module 250 in operation 340. The setof discrete content segments may be interspersed into the media streamin the order generated by the order module 250. In some embodiments, theset of discrete content segments may be interspersed into the mediastream by inserting the set of discrete content segments into the datastream representing the media stream such that the set of discretecontent segments becomes an integrated portion of the media stream. Inother instances, the set of discrete content segments may beinterspersed by substituting the set of discrete content segments forportions of the media stream without integrating the two streams. Forexample, the client device 110 may receive transmission of both thecontent stream and the media stream. At intervals during thetransmission of the media stream, the client device 110 may display oneor more of the set of discrete content segments. After display of theone or more discrete content segments, the client device 110 may resumedisplay of the media stream.

In some embodiments, the set of discrete content segments may beinterspersed into the media stream in subsets (e.g., bundles of multiplediscrete content segments) such that portions of the media stream arepresented between subsets of the set of discrete content segments. Insome instances, the set of discrete content segments may be dispersedwithin the media stream spaced apart by time intervals. The timeintervals may be evenly distributed during the transmission of the mediastream. In some instances, the time intervals may be dynamicallydistributed during the transmission of the media stream such that someof the time intervals may be shorter or longer than other timeintervals.

FIG. 4 is a flow chart of operations of the content ordering system 150in performing a method 400 of ordering content for transmission within acontent stream, according to some example embodiments. Operations in themethod 400 may be performed by the content ordering system 150, usingmodules described above with respect to FIG. 2. In some embodiments, theoperations of the method 400 can be performed using modules implementedat least in part on client devices, such as the client device 110, or onthe broadcast system 136.

The method 400 may be performed by initially executing operations 310,320, and 330, described above. In operation 410, the receiving module220 receives sensor data indicative of a set of engagement indicators.The engagement indicators may be understood as data indicating variouslevels of engagement of one or more subjects with the media stream. Theengagement indicators indicate the levels of engagement by representingactions of one or more subjects perceived by the sensor during displayof the media stream.

In some embodiments, the engagement indicators include a set of emotionindicators and a set of attention indicators. The emotion indicators canbe understood as actions, characteristics, or positions of a subjectindicative of an emotional state or response. As such, the emotionindicators may represent an emotional state of one or more subjectsperceived by the set of sensors at a display location. In someembodiments, the set of emotion indicators may be based on sensor datarepresenting one or more characteristics selected from a groupconsisting of a facial feature (e.g., a frown, a smile, a furrowed brow,wide eyes indicating surprise, etc.), a body position (e.g., posture,proximity of a body to the sensor, and the like), a volume ofconversation, a pace of conversation, a content of conversation, a lackof conversation, and the like.

The attention indicators may be understood as actions, characteristics,or positions of a subject indicative of the subject's level of attentiondirected to the media stream. These attention indicators may representwhether one or more subjects perceived by the set of sensors at thedisplay location are actively consuming, passively consuming, orignoring the media stream. Active consumption of a media stream can beunderstood as the one or more subjects paying direct attention to theassociated content, while passive consumption can be understood as theone or more subjects having their attention divided between the mediastream and another point of attention (e.g., another content stream, aconversation with another subject, a book, etc.). In some embodiments,the set of attention indicators may be based on sensor data representingone or more characteristics selected from a group comprising one or moreof a subject proximity to a sensor, a number of subjects perceived bythe sensor, a subject facial orientation (e.g., the subject positioningtheir face or eyes toward or away from the client device 110), a subjectbody orientation (e.g., a subject sitting down, a subject standing, asubject's shoulders positioned parallel to the client device 110, asubject's shoulders being at an angle to the client device 110, thesubject lying down, etc.), and other suitable characteristics capable ofindicating a focus of attention of the subject.

In some embodiments, the receiving module 220 may employ the set ofsensors 118 within or associated with the client device 110 to determinethe emotion indicators and the attention indicators. In someembodiments, the receiving module 220 may pass the raw sensor data,including the emotion indicators and the attention indicators, toanother module or application for analysis to identify the emotionindicators and attention indicators. For example, the receiving module220 may be in communication with the engagement module 240, which may becapable of providing facial recognition processes for identifying facialfeatures, expressions, and orientations of subjects perceived by the setof sensors 118. The engagement module 240 may also include voiceanalysis processes for determining conversation and conversation contentfrom general noise (e.g., ambient noise, cheering, etc.). The engagementmodule 240 may further include processes capable of determiningproximity of the subject to the set of sensors 118 and the number ofsubjects in the room. The engagement module 240 may also includeprocesses for pattern recognition to determine aspects andcharacteristics of the subjects perceived by the sensor. For example,the pattern recognition processes may determine wall decoration orjersey colors and patterns to determine a team affiliation of thesubjects consuming a media stream including a sporting event.

In operation 420, the engagement module 240 generates a set of emotionfactors based on the set of emotion indicators within the sensor data.The emotion factors may be understood as a set of data representative ofthe set of emotion indicators. The emotion factors are configured asinput into the order module 250 for determining an order for the set ofdiscrete content segments. In some embodiments, the emotion factors maybe a set of data values representing a facial feature, a volume ofconversation, a content of conversation, and the like received in thesensor data.

In operation 430, the engagement module 240 generates a set of attentionfactors based on the set of attention indicators within the sensor data.The attention factors may be understood as a set of data representativeof the set of attention indicators. The attention factors are configuredto provide input into the order module 250 for determining an order forthe set of discrete content segments. In some embodiments, the attentionfactors may be a set of data values representing a subject proximity tothe sensors, a number of subjects perceived by the sensor, a subjectfacial orientation relative to the sensor or the client device 110, asubject body orientation relative to the sensor or the client device110, or the like received in the sensor data.

In operation 440, the order module 250 generates an order for the set ofdiscrete content segments based on the set of event instances, the setof ordering factors for the set of discrete content segments, the set ofemotion factors, and the set of attention factors. In some embodiments,operation 440 may be implemented similarly to operation 340. Forexample, the order module 250 may reference a plurality of segmentmatrices mapping the set of discrete content segments to the orderingfactors and monetary values. The set of emotion factors and the set ofattention factors may be used in conjunction with the set of eventinstances to weight and order the set of discrete content segments basedon a determination of contextual appropriateness of a given discretecontent segment for the set of subjects consuming the media stream andthe events preceding presentation of the discrete content segment.

In some embodiments, the method 400 may include operation 450, in whichthe presentation module 260 causes presentation of the set of discretecontent segments within the media stream according to the orderdetermined by the order module 250 in operation 440. In someembodiments, operation 450 can be performed similarly to operation 350,described above.

FIG. 5 is a flow chart of operations of the content ordering system 150in performing a method 500 of ordering content for transmission within acontent stream, according to some example embodiments. Operations in themethod 500 may be performed by the content ordering system 150, usingmodules described above with respect to FIG. 2. In some embodiments, atleast a portion of one or more of the modules may be implemented on aclient device, such as the client device 110, or the broadcast system136.

In operation 510, the access module 210 accesses a set of discretecontent segments for display in a media stream. The access module 210may access the set of discrete content segments stored on the database126 or a database of the broadcast system 136 via the communicationmodule 270. In some embodiments, operation 510 may be performedsimilarly to or the same as operation 310.

In operation 520, the receiving module 220 receives first event datarepresentative of a first media stream. In some embodiments, the firstmedia stream may be a media stream in which the set of discrete contentsegments is to be presented. In some instances, the set of discretecontent segments may be distributed among a plurality of media streams.In some embodiments, operation 520 can be performed similarly tooperation 320.

In operation 530, the detection module 230 detects a set of first eventinstances from the first event data depicted by the first media stream.In some instances, operation 530 may be performed similarly to operation330.

In operation 540, the receiving module 220 receives second event datarepresentative of a second media stream. In some embodiments, the secondmedia stream may be a media stream being displayed at the same displaylocation as the first media stream, where the first media stream isdisplayed on a first client device and the second media stream isdisplayed on a second client device. In some instances, operation 540may be performed similarly to operation 320.

In some embodiments, the receiving module 220 may receive the secondevent data through the set of sensors associated with the client device110 displaying the first media stream. In these instances, the contentordering system 150 may have no direct access to the second mediastream, but receive the second event data through indirect means.

In operation 550, the detection module 230 detects a set of second eventinstances from the second event data depicted by the second mediastream. In some instances, operation 550 may be performed similarly tooperation 330.

In operation 560, the receiving module 220 receives sensor dataindicative of a set of engagement indicators. The engagement indicatorsmay be engagement indicators as described above with respect to FIG. 4.In some instances, one or more of the set of engagement indicators maybe indicative of attention directed to either the first media stream orthe second media stream. For example, a set of attention indicatorswithin the engagement indicators may indicate facial or body positioningof one or more subjects relative to the first client device displayingthe first media stream or the second client device displaying the secondmedia stream. In some embodiments, operation 560 may be performedsimilarly to operation 410, described above.

In operation 570, the engagement module 240 generates a first engagementscore for the first media stream and a second engagement score for thesecond media stream. In some embodiments, the engagement module 240generates a first set of emotion factors and a first set of attentionfactors for the first media stream and a second set of emotion factorsand a second set of attention factors for the second media stream. Theengagement module 240 can generate the first engagement score based onthe first set of emotion factors and the first set of attention factorsand the second engagement score based on the second set of emotionfactors and the second set of attention factors. In some embodiments,the first engagement score and the second engagement score are generatedrelative to one another to indicate whether the first media stream orthe second media stream is the primary focus of the one or moresubjects.

In operation 580, the order module 250 generates an order for the set ofdiscrete content segments based on the set of first event instances, theset of second event instances, the set of ordering factors for the setof discrete content segments, the first engagement score, and the secondengagement score. The order generated by the order module 250 may be asecond order where the set of discrete content segments is transmittedto the content ordering system 150 with a specified first order. In someembodiments, the order generated by the order module 250 may include aset of orders dividing the set of discrete content segments betweenmultiple client devices 110. In some embodiments, operation 580 may beperformed similarly to operation 340, described above.

In operation 590, the presentation module 260 causes presentation of theset of discrete content segments in one or more of the first mediastream and the second media stream. In at least some embodiments,operation 590 may be performed similarly to operation 350 or operation450.

Modules, Components, and Logic

Certain embodiments are described herein as including logic or a numberof components, modules, or mechanisms. Modules may constitute eithersoftware modules (e.g., code embodied on a machine-readable medium) orhardware modules. A “hardware module” is a tangible unit capable ofperforming certain operations and may be configured or arranged in acertain physical manner. In various example embodiments, one or morecomputer systems (e.g., a standalone computer system, a client computersystem, or a server computer system) or one or more hardware modules ofa computer system (e.g., a processor or a group of processors) may beconfigured by software (e.g., an application or application portion) asa hardware module that operates to perform certain operations asdescribed herein.

In some embodiments, a hardware module may be implemented mechanically,electronically, or any suitable combination thereof. For example, ahardware module may include dedicated circuitry or logic that ispermanently configured to perform certain operations. For example, ahardware module may be a special-purpose processor, such as aField-Programmable Gate Array (FPGA) or an Application SpecificIntegrated Circuit (ASIC). A hardware module may also includeprogrammable logic or circuitry that is temporarily configured bysoftware to perform certain operations. For example, a hardware modulemay include software executed by a general-purpose processor or otherprogrammable processor. Once configured by such software, hardwaremodules become specific machines (or specific components of a machine)uniquely tailored to perform the configured functions and are no longergeneral-purpose processors. It will be appreciated that the decision toimplement a hardware module mechanically, in dedicated and permanentlyconfigured circuitry, or in temporarily configured circuitry (e.g.,configured by software) may be driven by cost and time considerations.

Accordingly, the phrase “hardware module” should be understood toencompass a tangible entity, be that an entity that is physicallyconstructed, permanently configured (e.g., hardwired), or temporarilyconfigured (e.g., programmed) to operate in a certain manner or toperform certain operations described herein. As used herein,“hardware-implemented module” refers to a hardware module. Consideringembodiments in which hardware modules are temporarily configured (e.g.,programmed), each of the hardware modules need not be configured orinstantiated at any one instance in time. For example, where a hardwaremodule comprises a general-purpose processor configured by software tobecome a special-purpose processor, the general-purpose processor may beconfigured as respectively different special-purpose processors (e.g.,comprising different hardware modules) at different times. Softwareaccordingly configures a particular processor or processors, forexample, to constitute a particular hardware module at one instance oftime and to constitute a different hardware module at a differentinstance of time.

Hardware modules can provide information to, and receive informationfrom, other hardware modules. Accordingly, the described hardwaremodules may be regarded as being communicatively coupled. Where multiplehardware modules exist contemporaneously, communications may be achievedthrough signal transmission (e.g., over appropriate circuits and buses)between or among two or more of the hardware modules. In embodiments inwhich multiple hardware modules are configured or instantiated atdifferent times, communications between such hardware modules may beachieved, for example, through the storage and retrieval of informationin memory structures to which the multiple hardware modules have access.For example, one hardware module may perform an operation and store theoutput of that operation in a memory device to which it iscommunicatively coupled. A further hardware module may then, at a latertime, access the memory device to retrieve and process the storedoutput. Hardware modules may also initiate communications with input oroutput devices, and can operate on a resource (e.g., a collection ofinformation).

The various operations of example methods described herein may beperformed, at least partially, by one or more processors that aretemporarily configured (e.g., by software) or permanently configured toperform the relevant operations. Whether temporarily or permanentlyconfigured, such processors may constitute processor-implemented modulesthat operate to perform one or more operations or functions describedherein. As used herein, “processor-implemented module” refers to ahardware module implemented using one or more processors.

Similarly, the methods described herein may be at least partiallyprocessor-implemented, with a particular processor or processors beingan example of hardware. For example, at least some of the operations ofa method may be performed by one or more processors orprocessor-implemented modules. Moreover, the one or more processors mayalso operate to support performance of the relevant operations in a“cloud computing” environment or as a “software as a service” (SaaS).For example, at least some of the operations may be performed by a groupof computers (as examples of machines including processors), with theseoperations being accessible via a network (e.g., the Internet) and viaone or more appropriate interfaces (e.g., an Application ProgramInterface (API)).

The performance of certain of the operations may be distributed amongthe processors, not only residing within a single machine, but deployedacross a number of machines. In some example embodiments, the processorsor processor-implemented modules may be located in a single geographiclocation (e.g., within a home environment, an office environment, or aserver farm). In other example embodiments, the processors orprocessor-implemented modules may be distributed across a number ofgeographic locations.

Machine and Software Architecture

The modules, methods, applications, and so forth described inconjunction with FIGS. 1-5 are implemented in some embodiments in thecontext of a machine and an associated software architecture, a serverand an associated software architecture, or combinations thereof. Thesections below describe representative software architecture(s) andmachine (e.g., hardware) architecture(s) that are suitable for use withthe disclosed embodiments.

Software architectures are used in conjunction with hardwarearchitectures to create devices and machines tailored to particularpurposes. For example, a particular hardware architecture coupled with aparticular software architecture will create a mobile device, such as amobile phone, tablet device, or so forth. A slightly different hardwareand software architecture may yield a smart device for use in the“internet of things,” while yet another combination produces a servercomputer for use within a cloud computing architecture. Not allcombinations of such software and hardware architectures are presentedhere, as those of skill in the art can readily understand how toimplement the features of the present disclosure in different contextsfrom those presented herein.

Software Architecture

FIG. 6 is a block diagram 600 illustrating a representative softwarearchitecture 602, which may be used in conjunction with various hardwarearchitectures herein described. FIG. 6 is merely a non-limiting exampleof a software architecture, and it will be appreciated that many otherarchitectures may be implemented to facilitate the functionalitydescribed herein. The software architecture 602 may be executed onhardware such as a machine 700 of FIG. 7 that includes, among otherthings, processors 710, memory 730, and I/O components 750. Arepresentative hardware layer 604 is illustrated and can represent, forexample, the machine 700 of FIG. 7. The representative hardware layer604 comprises one or more processing units 606 having associatedexecutable instructions 608. The executable instructions 608 representthe executable instructions of the software architecture 602, includingimplementation of the methods, modules, and so forth of FIGS. 1-5. Thehardware layer 604 also includes memory and/or storage modules 610,which also contain the executable instructions 608. The hardware layer604 may also comprise other hardware 612, which includes any otherhardware of the hardware layer 604, such as the other hardwareillustrated as part of the machine 700.

In the example architecture of FIG. 6, the software architecture 602 maybe conceptualized as a stack of layers where each layer providesparticular functionality. For example, the software architecture 602 mayinclude layers such as an operating system 614, libraries 616,frameworks/middleware 618, applications 620, and a presentation layer644. Operationally, the applications 620 and/or other components withinthe layers may invoke application programming interface (API) calls 624through the software stack and receive a response, returned values, andso forth, illustrated as messages 626, in response to the API calls 624.The layers illustrated are representative in nature, and not allsoftware architectures have all layers. For example, some mobile orspecial purpose operating systems may not provide a layer offrameworks/middleware 618, while others may provide such a layer. Othersoftware architectures may include additional or different layers.

The operating system 614 may manage hardware resources and providecommon services. The operating system 614 may include, for example, akernel 628, services 630, and drivers 632. The kernel 628 may act as anabstraction layer between the hardware and the other software layers.For example, the kernel 628 may be responsible for memory management,processor management (e.g., scheduling), component management,networking, security settings, and so on. The services 630 may provideother common services for the other software layers. The drivers 632 maybe responsible for controlling or interfacing with the underlyinghardware. For instance, the drivers 632 may include display drivers,camera drivers, Bluetooth® drivers, flash memory drivers, serialcommunication drivers (e.g., Universal Serial Bus (USB) drivers), Wi-Fi®drivers, audio drivers, power management drivers, and so forth dependingon the hardware configuration.

The libraries 616 may provide a common infrastructure that may beutilized by the applications 620 and/or other components and/or layers.The libraries 616 typically provide functionality that allows othersoftware modules to perform tasks in an easier fashion than byinterfacing directly with the underlying operating system 614functionality (e.g., kernel 628, services 630, and/or drivers 632). Thelibraries 616 may include system libraries 634 (e.g., C standardlibrary) that may provide functions such as memory allocation functions,string manipulation functions, mathematic functions, and the like. Inaddition, the libraries 616 may include API libraries 636 such as medialibraries (e.g., libraries to support presentation and manipulation ofvarious media formats such as MPEG4, H.264, MP3, AAC, AMR, JPG, andPNG), graphics libraries (e.g., an OpenGL framework that may be used torender content in a 2D and 3D graphic context on a display), databaselibraries (e.g., SQLite that may provide various relational databasefunctions), web libraries (e.g., WebKit that may provide web browsingfunctionality), and the like. The libraries 616 may also include a widevariety of other libraries 638 to provide many other APIs to theapplications 620 and other software components/modules.

The frameworks/middleware 618 (also sometimes referred to as middleware)may provide a higher-level common infrastructure that may be utilized bythe applications 620 and/or other software components/modules. Forexample, the frameworks/middleware 618 may provide various graphic userinterface (GUI) functions, high-level resource management, high-levellocation services, and so forth. The frameworks/middleware 618 mayprovide a broad spectrum of other APIs that may be utilized by theapplications 620 and/or other software components/modules, some of whichmay be specific to a particular operating system or platform.

The applications 620 include built-in applications 640, third partyapplications 642, and/or instructions 643 for operating a portion of themethods described above to be performed by the content ordering system150. Examples of representative built-in applications 640 may include,but are not limited to, a contacts application, a browser application, abook reader application, a location application, a media application, amessaging application, and/or a game application. The third partyapplications 642 may include any of the built-in applications as well asa broad assortment of other applications. In a specific example, thethird party application 642 (e.g., an application developed using theAndroid™ or iOS™ software development kit (SDK) by an entity other thanthe vendor of the particular platform) may be mobile software running ona mobile operating system such as iOS™, Android™, Windows® Phone, orother mobile operating systems. In this example, the third partyapplication 642 may invoke the API calls 624 provided by the mobileoperating system such as the operating system 614 to facilitatefunctionality described herein.

The applications 620 may utilize built-in operating system functions(e.g., kernel 628, services 630, and/or drivers 632), libraries (e.g.,system 634, APIs 636, and other libraries 638), andframeworks/middleware (e.g., frameworks/middleware 618) to create userinterfaces to interact with users of the system. In some embodiments, aportion of the instructions 643 function as a part of theframeworks/middleware 618 to perform at least a portion of the methodsdescribed above to be performed by the content ordering system 150.Alternatively, or additionally, in some systems interactions with a usermay occur through a presentation layer, such as the presentation layer644. In these systems, the application/module “logic” can be separatedfrom the aspects of the application/module that interact with the user.

Some software architectures utilize virtual machines. In the example ofFIG. 6, this is illustrated by a virtual machine 648. A virtual machinecreates a software environment where applications/modules can execute asif they were executing on a hardware machine (such as the machine ofFIG. 7, for example). A virtual machine is hosted by a host operatingsystem (e.g., operating system 614 in FIG. 6) and typically, althoughnot always, has a virtual machine monitor 646, which manages theoperation of the virtual machine as well as the interface with the hostoperating system (e.g., operating system 614). A software architecture,which may include an operating system 650, libraries 652,frameworks/middleware 654, applications 656, and/or a presentation layer658, executes within the virtual machine 648. These layers of softwarearchitecture executing within the virtual machine 648 can be the same ascorresponding layers previously described or may be different.

Example Machine Architecture and Machine-Readable Medium

FIG. 7 is a block diagram illustrating components of a machine 700,according to some example embodiments, able to read instructions (e.g.,processor-executable instructions) from a machine-readable medium (e.g.,a non-transitory machine-readable storage medium) and perform any one ormore of the methodologies discussed herein. Specifically, FIG. 7 shows adiagrammatic representation of the machine 700 in the example form of acomputer system, within which instructions 716 (e.g., software, aprogram, an application, an applet, an app, or other executable code)for causing the machine 700 to perform any one or more of themethodologies discussed herein may be executed. For example, theinstructions 716 may cause the machine 700 to execute the flow diagramsof FIGS. 3-5. Additionally, or alternatively, the instructions 716 mayimplement the access module 210, the receiving module 220, the detectionmodule 230, the engagement module 240, the order module 250, thepresentation module 260, and the communication module 270 of FIG. 2, andso forth. The instructions transform the general, non-programmed machineinto a particular machine programmed to carry out the described andillustrated functions in the manner described. In alternativeembodiments, the machine 700 operates as a standalone device or may becoupled (e.g., networked) to other machines. In a networked deployment,the machine 700 may operate in the capacity of a server machine or aclient machine in a server-client network environment, or as a peermachine in a peer-to-peer (or distributed) network environment. Themachine 700 may comprise, but not be limited to, a server computer, aclient computer, a personal computer (PC), a tablet computer, a laptopcomputer, a netbook, a set-top box (STB), a personal digital assistant(PDA), an entertainment media system, a cellular telephone, a smartphone, a mobile device, a wearable device (e.g., a smart watch), a smarthome device (e.g., a smart appliance), other smart devices, a webappliance, a network router, a network switch, a network bridge, or anymachine 700 capable of executing the instructions 716, sequentially orotherwise, that specify actions to be taken by the machine 700. Further,while only a single machine 700 is illustrated, the term “machine” shallalso be taken to include a collection of machines 700 that individuallyor jointly execute the instructions 716 to perform any one or more ofthe methodologies discussed herein.

The machine 700 may include processors 710, memory/storage 730, and I/Ocomponents 750, which may be configured to communicate with each other,such as via a bus 702. In an example embodiment, the processors 710(e.g., a Central Processing Unit (CPU), a Reduced Instruction SetComputing (RISC) processor, a Complex Instruction Set Computing (CISC)processor, a Graphics Processing Unit (GPU), a Digital Signal Processor(DSP), an Application Specific Integrated Circuit (ASIC), aRadio-Frequency Integrated Circuit (RFIC), another processor, or anysuitable combination thereof) may include, for example, a processor 712and a processor 714 that may execute the instructions 716. The term“processor” is intended to include a multi-core processor that maycomprise two or more independent processors (sometimes referred to as“cores”) that may execute instructions contemporaneously. Although FIG.7 shows multiple processors, the machine 700 may include a singleprocessor with a single core, a single processor with multiple cores(e.g., a multi-core processor), multiple processors with a single core,multiple processors with multiples cores, or any combination thereof.

The memory/storage 730 may include a memory 732, such as a main memoryor other memory storage, and a storage unit 736, both accessible to theprocessors 710 such as via the bus 702. The storage unit 736 and memory732 store the instructions 716 embodying any one or more of themethodologies or functions described herein. The instructions 716 mayalso reside, completely or partially, within the memory 732, within thestorage unit 736, within at least one of the processors 710 (e.g.,within the processor's cache memory), or any suitable combinationthereof, during execution thereof by the machine 700. Accordingly, thememory 732, the storage unit 736, and the memory of the processors 710are examples of machine-readable media.

As used herein, “machine-readable medium” means a device able to storeinstructions and data temporarily or permanently, and may include, butis not limited to, random-access memory (RAM), read-only memory (ROM),buffer memory, flash memory, optical media, magnetic media, cachememory, other types of storage (e.g., Erasable Programmable Read-OnlyMemory (EEPROM)), and/or any suitable combination thereof. The term“machine-readable medium” should be taken to include a single medium ormultiple media (e.g., a centralized or distributed database, orassociated caches and servers) able to store the instructions 716. Theterm “machine-readable medium” shall also be taken to include anymedium, or combination of multiple media, that is capable of storinginstructions (e.g., instructions 716) for execution by a machine (e.g.,machine 700), such that the instructions, when executed by one or moreprocessors of the machine 700 (e.g., processors 710), cause the machine700 to perform any one or more of the methodologies described herein.Accordingly, a “machine-readable medium” refers to a single storageapparatus or device, as well as “cloud-based” storage systems or storagenetworks that include multiple storage apparatus or devices. The term“machine-readable medium” excludes signals per se.

The I/O components 750 may include a wide variety of components toreceive input, provide output, produce output, transmit information,exchange information, capture measurements, and so on. The specific I/Ocomponents 750 that are included in a particular machine will depend onthe type of machine. For example, portable machines such as mobilephones will likely include a touch input device or other such inputmechanisms, while a headless server machine will likely not include sucha touch input device. It will be appreciated that the I/O components 750may include many other components that are not shown in FIG. 7. The I/Ocomponents 750 are grouped according to functionality merely forsimplifying the following discussion, and the grouping is in no waylimiting. In various example embodiments, the I/O components 750 mayinclude output components 752 and input components 754. The outputcomponents 752 may include visual components (e.g., a display such as aplasma display panel (PDP), a light emitting diode (LED) display, aliquid crystal display (LCD), a projector, or a cathode ray tube (CRT)),acoustic components (e.g., speakers), haptic components (e.g., avibratory motor, resistance mechanisms), other signal generators, and soforth. The input components 754 may include alphanumeric inputcomponents (e.g., a keyboard, a touch screen configured to receivealphanumeric input, a photo-optical keyboard, or other alphanumericinput components), point based input components (e.g., a mouse, atouchpad, a trackball, a joystick, a motion sensor, or another pointinginstrument), tactile input components (e.g., a physical button, a touchscreen that provides location and/or force of touches or touch gestures,or other tactile input components), audio input components (e.g., amicrophone), and the like.

In further example embodiments, the I/O components 750 may includebiometric components 756, motion components 758, environmentalcomponents 760, or position components 762, among a wide array of othercomponents. For example, the biometric components 756 may includecomponents to detect expressions (e.g., hand expressions, facialexpressions, vocal expressions, body gestures, or eye tracking), measurebiosignals (e.g., blood pressure, heart rate, body temperature,perspiration, or brain waves), identify a person (e.g., voiceidentification, retinal identification, facial identification,fingerprint identification, or electroencephalogram basedidentification), and the like. The motion components 758 may includeacceleration sensor components (e.g., accelerometer), gravitation sensorcomponents, rotation sensor components (e.g., gyroscope), and so forth.The environmental components 760 may include, for example, illuminationsensor components (e.g., photometer), temperature sensor components(e.g., one or more thermometers that detect ambient temperature),humidity sensor components (not shown), pressure sensor components(e.g., barometer), acoustic sensor components (e.g., one or moremicrophones that detect background noise), proximity sensor components(e.g., infrared sensors that detect nearby objects) (not shown), gassensors (e.g., gas detection sensors to detect concentrations ofhazardous gases for safety or to measure pollutants in the atmosphere)(not shown), or other components that may provide indications,measurements, or signals corresponding to a surrounding physicalenvironment. The position components 762 may include location sensorcomponents (e.g., a Global Position System (GPS) receiver component),altitude sensor components (e.g., altimeters or barometers that detectair pressure from which altitude may be derived), orientation sensorcomponents (e.g., magnetometers), and the like.

Communication may be implemented using a wide variety of technologies.The I/O components 750 may include communication components 764 operableto couple the machine 700 to a network 780 or devices 770 via a coupling782 and a coupling 772 respectively. For example, the communicationcomponents 764 may include a network interface component or othersuitable device to interface with the network 780. In further examples,the communication components 764 may include wired communicationcomponents, wireless communication components, cellular communicationcomponents, Near Field Communication (NFC) components, Bluetooth®components (e.g., Bluetooth® Low Energy), Wi-Fi® components, and othercommunication components to provide communication via other modalities.The devices 770 may be another machine or any of a wide variety ofperipheral devices (e.g., a peripheral device coupled via a UniversalSerial Bus (USB)).

Moreover, the communication components 764 may detect identifiers orinclude components operable to detect identifiers. For example, thecommunication components 764 may include Radio Frequency Identification(RFID) tag reader components, NFC smart tag detection components,optical reader components (e.g., an optical sensor to detectone-dimensional bar codes such as Universal Product Code (UPC) bar code,multi-dimensional bar codes such as Quick Response (QR) code, Azteccode, Data Matrix, Dataglyph, MaxiCode, PDF417, Ultra Code, UCC RSS-2Dbar code, and other optical codes), or acoustic detection components(e.g., microphones to identify tagged audio signals). In addition, avariety of information may be derived via the communication components764, such as location via Internet Protocol (IP) geo-location, locationvia Wi-Fi® signal triangulation, location via detecting an NFC beaconsignal that may indicate a particular location, and so forth.

Transmission Medium

In various example embodiments, one or more portions of the network 780may be an ad hoc network, an intranet, an extranet, a virtual privatenetwork (VPN), a local area network (LAN), a wireless LAN (WLAN), a widearea network (WAN), a wireless WAN (WWAN), a metropolitan area network(MAN), the Internet, a portion of the Internet, a portion of the PublicSwitched Telephone Network (PSTN), a plain old telephone service (POTS)network, a cellular telephone network, a wireless network, a Wi-Fi®network, another type of network, or a combination of two or more suchnetworks. For example, the network 780 or a portion of the network 780may include a wireless or cellular network and the coupling 782 may be aCode Division Multiple Access (CDMA) connection, a Global System forMobile communications (GSM) connection, or another type of cellular orwireless coupling. In this example, the coupling 782 may implement anyof a variety of types of data transfer technology, such as SingleCarrier Radio Transmission Technology (1×RTT), Evolution-Data Optimized(EVDO) technology, General Packet Radio Service (GPRS) technology,Enhanced Data rates for GSM Evolution (EDGE) technology, thirdGeneration Partnership Project (3GPP) including 3G, fourth generationwireless (4G) networks, Universal Mobile Telecommunications System(UMTS), High Speed Packet Access (HSPA), Worldwide Interoperability forMicrowave Access (WiMAX), Long Term Evolution (LTE) standard, othersdefined by various standard-setting organizations, other long rangeprotocols, or other data transfer technology.

The instructions 716 may be transmitted or received over the network 780using a transmission medium via a network interface device (e.g., anetwork interface component included in the communication components764) and utilizing any one of a number of well-known transfer protocols(e.g., hypertext transfer protocol (HTTP)). Similarly, the instructions716 may be transmitted or received using a transmission medium via thecoupling 772 (e.g., a peer-to-peer coupling) to the devices 770. Theterm “transmission medium” shall be taken to include any intangiblemedium that is capable of storing, encoding, or carrying instructions716 for execution by the machine 700, and includes digital or analogcommunications signals or other intangible media to facilitatecommunication of such software.

Language

Throughout this specification, plural instances may implementcomponents, operations, or structures described as a single instance.Although individual operations of one or more methods are illustratedand described as separate operations, one or more of the individualoperations may be performed concurrently, and nothing requires that theoperations be performed in the order illustrated. Structures andfunctionality presented as separate components in example configurationsmay be implemented as a combined structure or component. Similarly,structures and functionality presented as a single component may beimplemented as separate components. These and other variations,modifications, additions, and improvements fall within the scope of thesubject matter herein.

Although an overview of the inventive subject matter has been describedwith reference to specific example embodiments, various modificationsand changes may be made to these embodiments without departing from thebroader scope of embodiments of the present disclosure. Such embodimentsof the inventive subject matter may be referred to herein, individuallyor collectively, by the term “invention” merely for convenience andwithout intending to voluntarily limit the scope of this application toany single disclosure or inventive concept if more than one is, in fact,disclosed.

The embodiments illustrated herein are described in sufficient detail toenable those skilled in the art to practice the teachings disclosed.Other embodiments may be used and derived therefrom, such thatstructural and logical substitutions and changes may be made withoutdeparting from the scope of this disclosure. The Detailed Description,therefore, is not to be taken in a limiting sense, and the scope ofvarious embodiments is defined only by the appended claims, along withthe full range of equivalents to which such claims are entitled.

As used herein, the term “or” may be construed in either an inclusive orexclusive sense. Moreover, plural instances may be provided forresources, operations, or structures described herein as a singleinstance. Additionally, boundaries between various resources,operations, modules, engines, and data stores are somewhat arbitrary,and particular operations are illustrated in a context of specificillustrative configurations. Other allocations of functionality areenvisioned and may fall within a scope of various embodiments of thepresent disclosure. In general, structures and functionality presentedas separate resources in the example configurations may be implementedas a combined structure or resource. Similarly, structures andfunctionality presented as a single resource may be implemented asseparate resources. These and other variations, modifications,additions, and improvements fall within a scope of embodiments of thepresent disclosure as represented by the appended claims. Thespecification and drawings are, accordingly, to be regarded in anillustrative rather than a restrictive sense.

What is claimed is:
 1. A method comprising: receiving, by at least oneprocessor of a machine, a set of content segments having a first orderfor distributing the set of content segments during a display of a mediastream comprising media distinct from the set of content segments;receiving event data representative of a set of events depicted by themedia comprising the media stream; determining, using the at least oneprocessor of the machine, from the event data, a set of event instancesof the set of events depicted by the media of the media stream; andgenerating, using the at least one processor of the machine, a secondorder for the set of content segments responsive to determining at leastone event instance as the event data is received, the second order beingdistinct from the first order and periodically changing based on the setof event instances represented by the event data.
 2. The method of claim1, further comprising: accessing sensor data depicting one or moreaudience members of the media stream; and based on the sensor data,determining, using the at least one processor of the machine, anemotional state of at least one audience member of the one or moreaudience members.
 3. The method of claim 2, wherein generating thesecond order for the set of content segments is based on the set ofevent instances and the emotional state of the at least one audiencemember.
 4. The method of claim 2, wherein the emotional state isdetermined based on one or more of a facial expression and a bodyposition.
 5. The method of claim 2, wherein the sensor data includesinformation representing one or more objects perceived within a field ofview along with the at least one audience member, the method furthercomprising: determining an affiliation associated with the one or moreobjects perceived in the field of view, the affiliation being betweenone or more objects, the at least one audience member, and a portion ofthe event data; and determining the emotional state based on the sensordata depicting the one or more audience member, the affiliation, and atleast a portion of the event data.
 6. The method of claim 2, wherein thesensor data depicting one or more audience members is first sensor data,the method further comprising: accessing second sensor data comprisingat least a portion of the event data presented within the media stream;determining the first sensor data is in response to the second sensordata; and based on the first sensor data and the second sensor data,determining the emotional state of the at least one audience members. 7.The method of claim 1, wherein receiving the event data furthercomprises: accessing sensor data comprising at least a portion of theevent data presented within the media stream; and determining the sensordata includes an action directly related to the media stream.
 8. Themethod of claim 1, wherein receiving the event data thither comprises:accessing sensor data comprising a conversation including one or moreaudience member of the media stream; determining a content of theconversation relating to the media stream; and generating a set ofattention factors based on the content of the conversation.
 9. A system,comprising: one or more processors; and a machine-readable storagedevice comprising instructions that, when executed by the one or moreprocessors, cause the one or more processors to perform operationscomprising: receiving a set of content segments having a first order fordistributing the set of content segments during a display of a mediastream comprising media distinct from the set of content segments;receiving event data representative of a set of events depicted by themedia comprising the media stream; determining from the event data, aset of event instances of the set of events depicted by the media of themedia stream; and generating a second order for the set of contentsegments responsive to determining at least one event instance as theevent data is received, the second order being distinct from the firstorder and periodically changing based on the set of event instancesrepresented by the event data.
 10. The system of claim 9, wherein theoperations further comprise: accessing sensor data depicting one or moreaudience members of the media stream; and based on the sensor data,determining, using the processor of the machine, an emotional state ofat least one audience member of the one or more audience members. 11.The system of claim 10, wherein generating the second order for the setof content segments is based on the set of event instances and theemotional state of the at least one audience member.
 12. The system ofclaim 10, wherein the emotional state is determined based on one or moreof a facial expression and a body position.
 13. The system of claim 10,wherein the sensor data includes information representing one or moreobjects perceived within a field of view along with the at least oneaudience member, the operations further comprising: determining anaffiliation associated with the one or more objects perceived in thefield of view, the affiliation being between one or more objects, the atleast one audience member, and a portion of the event data; anddetermining the emotional state based on the sensor data depicting theone or more audience member, the affiliation, and at least a portion ofthe event data.
 14. The system of claim 10, wherein the sensor datadepicting one or more audience members is first sensor data, theoperations further comprising: accessing second sensor data comprisingat least a portion of the event data presented within the media stream;determining the first sensor data is in response to the second sensordata; and based on the first sensor data and the second sensor data,determining the emotional state of the at least one audience members.15. The system of claim 9, wherein receiving the event data furthercomprises: accessing sensor data comprising a conversation including oneor more audience member of the media stream; determining a content ofthe conversation relating to the media stream; and generating a set ofattention factors based on the content of the conversation.
 16. Amachine-readable storage device comprising instructions that, whenexecuted by one or more processors of a machine, cause the machine toperform operations comprising: receiving a set of content segmentshaving a first order for distributing the set of content segments duringa display’ of a media stream comprising media distinct from the set ofcontent segments; receiving event data representative of a set of eventsdepicted by the media comprising the media stream; determining from theevent data, a set of event in stances of the set of events depicted bythe media of the media stream; and generating a second order for the setof content segments responsive to determining at least one eventinstance as the event data is received, the second order being distinctfrom the first order and periodically changing based on the set of eventinstances represented by the event data.
 17. The machine-readablestorage device of claim 16, wherein the operations further comprise:accessing sensor data depicting one or more audience members of themedia stream; and based on the sensor data, determining, using theprocessor of the machine, an emotional state of at least one audiencemember of the one or more audience members.
 18. The machine-readablestorage device of claim 17, wherein the sensor data includes informationrepresenting one or more objects perceived within a field of view alongwith the at least one audience member, the operations furthercomprising: determining an affiliation associated with the one or moreobjects perceived in the field of view, the affiliation being betweenone or more objects, the at least one audience member, and a portion ofthe event data; and determining the emotional state based on the sensordata depicting the one or more audience member, the affiliation, and atleast a portion of the event data.
 19. The machine-readable storagedevice of claim 17, wherein the sensor data depicting one or moreaudience members is first sensor data, the operations furthercomprising: accessing second sensor data comprising at least a portionof the event data presented within the media stream; determining thefirst sensor data is in response to the second sensor data; and based onthe first sensor data and the second sensor data, determining theemotional state of the at least one audience members.
 20. Themachine-readable storage device of claim 17, wherein receiving the eventdata further comprises: accessing sensor data comprising a conversationincluding one or more audience member of the media stream; determining acontent of the conversation relating to the media stream; and generatinga set of attention factors based on the content of the conversation.